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(57)Abstract: 

PURPOSE: To transmit in real time the detailed 
contents of news to the aurally handicapped persons 
with no hand language expert needed. 
CONSTITUTION: A data register part 5 registers the lip 
patterns received from a camera 1 and the voice 
waveforms received from a microphone 2 before a news 
program is transmitted. A paragraph storing part 6 
recognizes the contents registered in the part 5 and 
converts this recognition result into the paragraph 
information to store it. A data collating part 7 collates 
the contents of the part 5 with the lip pattern 
recognized by a pattern recognizing part 3 and the voice 
waveform recognized toy a voice recognizing part 4 
respectively and controls a paragraph transmitting part 8 
and a paragraph converting part 9 based on the collation 
results. A signal converter 10 converts the paragraph 
information received from both parts 8 and 9 into the 
broadcast signals. Then a signal synthesizer 1 1 
superposes the broadcast signals converted by the 

converter 10 on the transmission screen of the news program and transmits these broadcast 
signals. 
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* NOTICES * 

JPO and NCIPI are not responsible for any 
damages caused by the use of this translation. 

LThis document has been translated by computer. So the translation may not reflect the original 
precisely. 

2.**** shows the word which can not be translated. 
3.1n the drawings, any words are not translated. 



DETAILED DESCRIPTION 



[Detailed Description of the Invention] 
[0001] 

[Industrial Application] Especially this invention relates to sending out of the news program in 
broadcast operation business about the news sending-out equipment for hearing-impaired 
persons. 
[0002] 

[Description of the Prior Art] Conventionally, in news sending out in a TV program, there are an 
approach of projecting the person who performs sign language in a corner of the television 
screen which displays announcer's etc. image, and the approach of using as an alphabetic 
character screen only the main point of the news prepared beforehand, and compounding on a 
television screen as a program for hearing-impaired persons. 

[0003] In these approaches, although the method of projecting the person who performs sign 
language in a corner of a television screen chiefly on coincidence is used when transmitting the 
detailed contents of news to a hearing-impaired person, the expert of sign language is needed 
using this approach. 

[0004] When it is going to transmit the detailed contents of news without the expert of sign 
language to a hearing-impaired person, the contents of the news are recognized using a voice 
recognition unit etc., and how to compound the recognition result on a television screen can be 
considered. 

[0005] The approach of making a data input possible by recognizing a motion of an utterance 
person's opening like the technique indicated by JP,1-230123,A as a voice recognition unit which 
can be used by the above-mentioned approach, the method of associating an utterance person's 
voice and lips configuration like the technique indicated by JP,1 -25941 4,A, and performing 
speech recognition, etc. are proposed. 

[0006] Moreover, the method of recognizing voice like the technique indicated by JP,1-259414,A 
by adding the information by correlation with an utterance sound and a motion of opening using 
the information processing technique is also proposed. 
[0007] 

[Problem(s) to be Solved by the Invention] In news sending out in the conventional TV program 
mentioned above, since the contents of news are transmitted to the hearing-impaired person by 
using the approach of using as an alphabetic character screen only the main point of the method 
of projecting the person who performs sign language in a corner of a television screen on 
coincidence, or news, and compounding on a television screen, while needing the expert of sign 
language, if there is not an expert of sign language, only the main point of news can be 
transmitted. 

[0008] In order to solve this, displaying the contents which used the above-mentioned voice 
recognition unit and have been recognized from an utterance person's voice and lips 
configuration on a television screen is also considered, but since high-speed processing is 
needed for the speech recognition, it is difficult to generate the alphabetic character screen 
which should be displayed with the image of a television screen on real time. 
[0009] Then, it is in the purpose of this invention offering the news sending-out equipment for 
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the hearing-impaired persons who can transmit the detailed contents of news to real time to a 
hearing-impaired person, without canceling the above-mentioned trouble and needing the expert 
of sign language. 
[0010] 

[Means for Solving the Problem] The 1st news sending-out equipment for trouble back tone by 
this invention A collating means to collate whether it is in the lips pattern with which lips pattern 
and voice of the utterance person who reads out the contents of news were beforehand 
registered at the time of news sending out, and voice, The 1st recognition means which 
recognizes an utterance person's lips pattern and voice at the time of said news sending out, 
and acquires text, The 2nd recognition means which recognizes the lips pattern and voice which 
were registered beforehand, and acquires text, a conversion means to change one side into the 
signal for broadcast according to the collating result of said collating means among said 1 st and 
2nd recognition results of each recognition means, and a synthetic means to superimpose the 
conversion result of said conversion means on a news sending-out signal are provided. 
[001 1] The 2nd news sending-out equipment for hearing-impaired persons by this invention A 
photography means to photo an utterance person's lips pattern, and a collection means to 
collect said utterance person's voice, A registration means to register the voice wave by the lips 
pattern by said photography means, and said collection means, A preservation means to 
recognize said lips pattern registered into said registration means, and said voice wave, and to 
change and save at text, A lips pattern recognition means to recognize said lips pattern and to 
change into text, A collating means to collate a speech recognition means to recognize said 
voice wave and to change into text, and the lips pattern of the utterance person who reads out 
the contents of news at the time of news sending out, and a voice wave and the contents of said 
registration means, a conversion means to change one side into the signal for broadcast 
according to the collating result of said collating means among the contents of said lips pattern 
recognition means, and said recognition result of each speech recognition means and said 
preservation means, and a synthetic means to superimpose the conversion result of said 
conversion means on a news sending-out signal are provided. 

[0012] The 3rd news sending-out equipment for hearing-impaired persons by this invention A 
lips pattern recognition means to recognize an utterance person's lips pattern and to change into 
text, with a speech recognition means to recognize said utterance person's voice and to change 
into text, and a registration means to register said lips pattern recognition means and said 
recognition result of each speech recognition means with a collating means to collate the 
contents of said lips pattern recognition means against lips pattern and voice of the utterance 
person who reads out the contents of news at the time of news sending out, and said recognition 
result of each speech recognition means and said registration means it has a conversion means 
to change one side into the signal for broadcast according to the collating result of said collating 
means among the contents of said lips pattern recognition means, and said recognition result of 
each speech recognition means and said registration means, and a synthetic means to 
superimpose the conversion result of said conversion means on a news sending-out signal. 
[0013] The 4th news sending-out equipment for hearing-impaired persons by this invention A 
lips pattern recognition means to recognize an utterance person's lips pattern and to change into 
text, with a speech recognition means to recognize said utterance person's voice and to change 
into text, and a clause signal transduction means to change said lips pattern recognition means 
and said recognition result of each speech recognition means into clause information A collating 
means to collate the conversion result of a registration means to register the conversion result 
of said clause signal transduction means, and said clause signal transduction means against lips 
pattern and voice of the utterance person who reads out the contents of news at the time of 
news sending out, and the contents of said registration means, It has a signal transformation 
means to change one side into the signal for broadcast according to the collating result of said 
collating means among the conversion result of said clause signal transduction means, and the 
contents of said registration means, and a synthetic means to superimpose the conversion result 
of said signal transformation means on a news sending-out signal. 

[0014] The 5th news sending-out equipment for hearing-impaired persons by this invention A 
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photography means to photo an utterance person's lips pattern, and a collection means to 
collect said utterance person's voice, A registration means to register the voice wave by the lips 
pattern by said photography means, and said collection means, A preservation means to 
recognize said lips pattern registered into said registration means, and said voice wave, and to 
change and save the recognition result to clause information, A lips pattern recognition means to 
recognize said lips pattern and to change into text, A collating means to collate a speech 
recognition means to recognize said voice wave and to change into text, and the lips pattern of 
the utterance person who reads out the contents of news at the time of news sending out, and a 
voice wave and the contents of said registration means, The 1st signal transformation means 
which changes the contents of said preservation means into the signal for broadcast when the 
collating result of said collating means shows coincidence, with a clause conversion means to 
change said lips pattern recognition means and said recognition result of each speech 
recognition means into clause information, when the collating result of said collating means 
shows an inequality it has the 2nd signal transformation means which changes the conversion 
result of said clause conversion means into said signal for broadcast, and a synthetic means to 
superimpose said 1 st and 2nd conversion results of each signal transformation means on a news 
sending-out signal. 

[0015] The 6th news sending-out equipment for hearing-impaired persons by this invention A 
lips pattern recognition means to recognize an utterance person's lips pattern and to change into 
text, with a speech recognition means to recognize said utterance person's voice and to change 
into text, and a registration means to register said lips pattern recognition means and said 
recognition result of each speech recognition means with a collating means to collate the 
contents of a preservation means to change and save the contents of said registration means to 
clause information, said lips pattern recognition means against lips pattern and voice of the 
utterance person who reads out the contents of news at the time of news sending out, and said 
recognition result of each speech recognition means and said registration means it has a 
conversion means to change one side into the signal for broadcast according to the collating 
result of said collating means among the contents of said lips pattern recognition means, and 
said recognition result of each speech recognition means and said preservation means, and a 
synthetic means to superimpose the conversion result of said conversion means on a news 
sending-out signal. 
[0016] 

[Function] The lips pattern of the utterance person at the time of the rehearsal before news 
program sending out and a voice wave are beforehand registered into the data registration 
section. At this time, pattern recognition and speech recognition are performed to the contents 
registered into the data registration section, and it is further changed into clause information, 
and is saved in the clause preservation section. 

[0017] An utterance person's lips pattern and a voice wave are collated with the contents 
registered into the data registration section in the data collating section at the time of news 
sending out. Among the clause information saved in the clause preservation section according to 
this collating result, and the clause information which changed the pattern recognition result of 
an utterance person's lips pattern, and the audio speech recognition result, it is changed into the 
signal for broadcast with a signal converter, and with a signal composition vessel, the sending- 
out screen of a news program is overlapped on one side, and it is sent out to it. 
[0018] By this, transfer on real time is attained to a hearing-impaired person in the detailed 
contents of news, without needing the expert of sign language. 
[0019] 

[Example] Next, this invention is explained with reference to a drawing. 

[0020] Drawing 1 is the block diagram showing the configuration of one example of this invention. 
In drawing, the news sending-out equipment for hearing-impaired persons by one example of this 
invention consists of a camera 1, a microphone 2, the pattern recognition section 3, the speech 
recognition section 4, the data registration section 5, the clause preservation section 6, the data 
collating section 7, the clause sending-out section 8, a clause transducer 9, a signal converter 
10, and a signal composition machine 11. 



http://www4.ipdl.ncipi.go.jp/cgi-bin/tran_web_cgi_ejje 



2005/11/28 



JP,08-009254,A [DETAILED DESCRIPTION] 



4/9 s<— v 



[0021] A camera 1 photos the lips pattern of utterance persons, such as announcer, and sends 
out the photoed lips pattern to the pattern recognition section 3 and the data registration 
section 5. A microphone 2 collects the voice of utterance persons, such as announcer, and 
sends out the collected voice waves to the speech recognition section 4 and the data 
registration section 5. 

[0022] The pattern recognition section 3 performs pattern recognition to the lips pattern 
photoed with the camera 1 , and outputs the text which it is as a result of the pattern recognition 
to the data collating section 7. The speech recognition section 4 performs speech recognition to 
the voice collected with the microphone 2, and outputs the text which it is as a result of the 
speech recognition to the data collating section 7. 

[0023] The voice wave from the lips pattern and microphone 2 from a camera 1 is beforehand 
registered into the data registration section 5. The clause preservation section 6 performs 
pattern recognition and speech recognition to the lips pattern and voice wave which were 
beforehand registered into the data registration section 5, respectively, and changes and saves 
the text which it is as a result of [ the ] recognition to clause information. 
[0024] If the data collating section 7 performs collating with the voice wave by which speech 
recognition is carried out in the lips pattern and the speech recognition section 4 by which 
pattern recognition is carried out in the pattern recognition section 3, and the contents 
beforehand registered into the data registration section 5 and its they correspond, it will direct 
to send out the clause information saved in the clause sending-out section 8 at the clause 
preservation section 6. 

[0025] On the other hand, if the data collating section 7 has inharmonious them, it will direct to 
change the result of the pattern recognition of the pattern recognition section 3, and the result 
of the speech recognition of the speech recognition section 4 into clause information to the 
clause transducer 9. 

[0026] The clause sending-out section 8 sends out the clause information which answered 
directions from the data collating section 7, and was saved in the clause preservation section 6 
to a signal converter 10. The clause transducer 9 answers directions from the data collating 
section 7, changes the result of the pattern recognition of the pattern recognition section 3, and 
the result of the speech recognition of the speech recognition section 4 into clause information, 
and sends them out to a signal converter 10. 

[0027] A signal converter 10 changes the clause information from the clause sending-out 
section 8 and the clause transducer 9 into the signals for broadcast (NTSC signal etc.), and 
outputs it to the signal composition machine 1 1 according to an office synchronizing signal. 
[0028] The signal composition machine 1 1 superimposes the signal for broadcast changed into 
the screen in which announcer etc. was reflected, i.e., the screen sent out in a news program, 
with the signal converter 10, and sends this out as a sending-out screen. 

[0029] Actuation of one example of this invention is explained using this drawing 1 . First, when 
announcer etc. performs a rehearsal before sending out a news program, while photoing the lips 
pattern of utterance persons, such as announcer, with a camera 1 , the voice of utterance 
persons, such as announcer, is collected with a microphone 2. 

[0030] The voice collected with the microphone 2 as if the lips pattern photoed with the camera 
1 is sent out to the pattern recognition section 3 and the data registration section 5 is sent out 
to the speech recognition section 4 and the data registration section 5. 
[0031] The data registration section 5 registers the voice wave from the lips pattern and 
microphone 2 from a camera 1 . At this time, the clause preservation section 6 performs pattern 
recognition and speech recognition to the lips pattern and voice wave which were registered into 
the data registration section 5, and changes and saves the text which it is as a result of [ that ] 
recognition to clause information. 

[0032] If a microphone 2 collects the voice of utterance persons, such as announcer, while a 
camera 1 photos the lips pattern of utterance persons, such as announcer, at the time of 
sending out of a news program, the pattern recognition section 3 will perform pattern recognition 
to the lips pattern from a camera 1 , and the speech recognition section 4 will perform speech 
recognition to the voice from a microphone 2. 
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[0033] If the data collating section 7 performs collating with the voice wave which carries out 
speech recognition in the lips pattern and the speech recognition section 4 which carry out 
pattern recognition in the pattern recognition section 3, and the contents of the data registration 
section 5 and its they correspond, it will direct to send out the clause information saved in the 
clause preservation section 6 to the clause sending-out section 8 to a signal converter 10. 
[0034] The clause sending-out section 8 sends out the clause information which answered 
directions from the data collating section 7, and was saved in the clause preservation section 6 
to a signal converter 10. A signal converter 10 changes the clause information from the clause 
sending-out section 8 into the signal for broadcast and outputs it to the signal composition 
machine 1 1 according to an office synchronizing signal. 

[0035] The signal composition machine 1 1 superimposes the signal for broadcast changed into 
the screen in which announcer etc. was reflected, i.e., the screen sent out in a news program, 
with the signal converter 10, and sends this out as a sending-out screen. 
[0036] On the other hand, the data collating section 7 will direct to change the result of the 
pattern recognition of the pattern recognition section 3, and the result of the speech recognition 
of the speech recognition section 4 into clause information to the clause transducer 9, if the 
voice wave which carries out speech recognition in the lips pattern and the speech recognition 
section 4 which carry out pattern recognition in the pattern recognition section 3, and the 
contents of the data registration section 5 are inharmonious (i.e., if it is different contents the 
time of the contents of news being rehearsals (for example, a news bulletin, relay broadcast, 
etc.)). 

[0037] The clause transducer 9 answers directions from the data collating section 7, changes 
the result of the pattern recognition of the pattern recognition section 3, and the result of the 
speech recognition of the speech recognition section 4 into clause information, and sends them 
out to a signal converter 10. A signal converter 10 changes the clause information from the 
clause transducer 9 into the signal for broadcast, and outputs it to the signal composition 
machine 1 1 according to an office synchronizing signal. 

[0038] The signal composition machine 1 1 superimposes the signal for broadcast changed into 
the screen in which announcer etc. was reflected, i.e., the screen sent out in a news program, 
with the signal converter 1 0, and sends this out as a sending-out screen. 
[0039] Drawing 2 is the block diagram showing the configuration of other examples of this 
invention. In drawing, the news sending-out equipment for hearing-impaired persons by other 
examples of this invention The result of the pattern recognition of the pattern recognition 
section 3 and the result of the speech recognition of the speech recognition section 4 are 
beforehand registered into the data registration section 1 2. Except having collated the 
recognition result of the pattern recognition section 3 and the speech recognition section 4, and 
the contents of the data registration section 12 in the data collating section 14, it has the same 
composition as one example of this invention shown in drawing 1 R> 1, and the same sign is 
given to the same component. Moreover, processing actuation of these same components is the 
same as that of one example of this invention. 

[0040] Here, since the data collating section 14 only collates the text from the pattern 
recognition section 3 and the speech recognition section 4, and the text of the data registration 
section 12, processing actuation can be made into a high speed compared with the above- 
mentioned lips pattern and collating of a voice wave. 

[0041] Moreover, since the clause preservation section 13 only changes into clause information 
the text registered into the data registration section 12, processing actuation can be made into a 
high speed like the data collating section 14. 

[0042] Actuation of one example of this invention is explained using this drawing 2 . First, when 
announcer etc. performs a rehearsal before sending out a news program, while photoing the lips 
pattern of utterance persons, such as announcer, with a camera 1, the voice of utterance 
persons, such as announcer, is collected with a microphone 2. 

[0043] The voice collected with the microphone 2 as if the lips pattern photoed with the camera 
1 is sent out to the pattern recognition section 3 is sent out to the speech recognition section 4. 
The pattern recognition section 3 performs pattern recognition to the lips pattern from a camera 
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1 , and sends out the recognition result to the data registration section 1 2 and the data collating 
section 14. 

[0044] The speech recognition section 4 performs speech recognition to the voice from a 
microphone 2, and sends out the recognition result to the data registration section 1 2 and the 
data collating section 14. 

[0045] The data registration section 1 2 registers beforehand the recognition result of the pattern 
recognition section 3 and the speech recognition section 4. At this time, the clause preservation 
section 1 3 changes and saves to clause information as a result of [ of the pattern recognition 
section 3 registered into the data registration section 1 2, and the speech recognition section 4 ] 
recognition (i.e., the text which it is as a result of [ that ] recognition). 

[0046] If a microphone 2 collects the voice of utterance persons, such as announcer, while a 
camera 1 photos the lips pattern of utterance persons, such as announcer, at the time of 
sending out of a news program, the pattern recognition section 3 will perform pattern recognition 
to the lips pattern from a camera 1 , and the speech recognition section 4 will perform speech 
recognition to the voice from a microphone 2. 

[0047] If the data collating section 14 performs collating with the recognition result of the 
pattern recognition section 3 and the speech recognition section 4, and the contents of the data 
registration section 12 and its they correspond, it will direct to send out the clause information 
saved in the clause preservation section 1 3 to the clause sending-out section 8 to a signal 
converter 10. 

[0048] The clause sending-out section 8 sends out the clause information which answered 
directions from the data collating section 14, and was saved in the clause preservation section 
13 to a signal converter 10. A signal converter 10 changes the clause information from the 
clause sending-out section 8 into the signal for broadcast, and outputs it to the signal 
composition machine 1 1 according to an office synchronizing signal. 

[0049] The signal composition machine 1 1 superimposes the signal for broadcast changed into 
the screen in which announcer etc. was reflected, i.e., the screen sent out in a news program, 
with the signal converter 1 0, and sends this out as a sending-out screen. 
[0050] On the other hand, the data collating section 14 will direct to change the recognition 
result of the pattern recognition section 3 and the speech recognition section 4 into clause 
information to the clause transducer 9, if the recognition result of the pattern recognition 
section 3 and the speech recognition section 4 and the contents of the data registration section 
1 2 are inharmonious (i.e., if it is different contents the time of the contents of news being 
rehearsals (for example, a news bulletin, relay broadcast, etc.)). 

[0051] The clause transducer 9 answers directions from the data collating section 14, changes 
the recognition result of the pattern recognition section 3 and the speech recognition section 4 
into clause information, and sends it out to a signal converter 10. A signal converter 10 changes 
the clause information from the clause transducer 9 into the signal for broadcast, and outputs it 
to the signal composition machine 1 1 according to an office synchronizing signal. 
[0052] The signal composition machine 1 1 superimposes the signal for broadcast changed into 
the screen in which announcer etc. was reflected, i.e., the screen sent out in a news program, 
with the signal converter 10, and sends this out as a sending-out screen. 
[0053] Drawing 3 is the block diagram showing the configuration of another example of this 
invention. In drawing, the news sending-out equipment for hearing-impaired persons by another 
example of this invention The clause information changed into the clause registration section 16 
by the clause transducer 15 is registered beforehand. Except having collated the head part of 
clause information and the contents of the clause registration section 1 6 which were changed by 
the clause transducer 15 in the data collating section 17, it has the same composition as one 
example of this invention shown in drawing 1 , and the same sign is given to the same 
component. Moreover, processing actuation of these same components is the same as that of 
one example of this invention. 

[0054] Here, since the clause collating section 1 7 only collates the head part of the clause 
information from the clause transducer 1 5, and the clause information on the clause registration 
section 16, processing actuation can be made into a high speed compared with the above- 
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mentioned lips pattern and collating of a voice wave. 

[0055] Actuation of one example of this invention is explained using this drawing 3 . First, when 
announcer etc. performs a rehearsal before sending out a news program, while photoing the lips 
pattern of utterance persons, such as announcer, with a camera 1 , the voice of utterance 
persons, such as announcer, is collected with a microphone 2. 

[0056] The voice collected with the microphone 2 as if the lips pattern photoed with the camera 
1 is sent out to the pattern recognition section 3 is sent out to the speech recognition section 4. 
The pattern recognition section 3 performs pattern recognition to the lips pattern from a camera 
1, and sends out the recognition result to the clause transducer 15. 

[0057] The speech recognition section 4 performs speech recognition to the voice from a 
microphone 2, and sends out the recognition result to the clause transducer 1 5. The clause 
transducer 15 changes the recognition result of the pattern recognition section 3 and the 
speech recognition section 4 into clause information, and sends out the clause information to the 
clause registration section 1 6 and the clause collating section 1 7. The clause registration section 
16 registers beforehand the clause information changed by the clause transducer 15. 
[0058] If a microphone 2 collects the voice of utterance persons, such as announcer, while a 
camera 1 photos the lips pattern of utterance persons, such as announcer, at the time of 
sending out of a news program, the pattern recognition section 3 will perform pattern recognition 
to the lips pattern from a camera 1 , and the speech recognition section 4 will perform speech 
recognition to the voice from a microphone 2. 

[0059] The clause transducer 1 5 changes the recognition result of the pattern recognition 
section 3 and the speech recognition section 4 into clause information, and sends out the clause 
information to the clause collating section 1 7. If the clause collating section 1 7 performs 
collating with the head part of clause information and the contents of the clause registration 
section 16 which were changed by the clause transducer 15 and its they correspond, it will 
direct to send out the clause information registered into the clause registration section 16 to the 
clause sending-out section 8 to a signal converter 10. 

[0060] The clause sending-out section 8 sends out the clause information which answered 
directions from the clause collating section 17, and was registered into the clause registration 
section 16 to a signal converter 10. A signal converter 10 changes the clause information from 
the clause sending-out section 8 into the signal for broadcast, and outputs it to the signal 
composition machine 1 1 according to an office synchronizing signal. 

[0061] The signal composition machine 1 1 superimposes the signal for broadcast changed into 
the screen in which announcer etc. was reflected, i.e., the screen sent out in a news program, 
with the signal converter 10, and sends this out as a sending-out screen. 
[0062] On the other hand, the clause collating section 17 sends out the clause information 
changed by the clause transducer 15 to a signal converter 10, if the head part of clause 
information and the contents of the clause registration section 16 which were changed by the 
clause transducer 15 are inharmonious (i.e., if the times of the contents of news being rehearsals 
are different contents (for example, a news bulletin, relay broadcast, etc.)). 

[0063] A signal converter 10 changes the clause information from the clause collating section 17 
into the signal for broadcast, and outputs it to the signal composition machine 1 1 according to 
an office synchronizing signal. The signal composition machine 1 1 superimposes the signal for 
broadcast changed into the screen in which announcer etc. was reflected, i.e., the screen sent 
out in a news program, with the signal converter 10, and sends this out as a sending-out screen. 
[0064] Thus, an utterance person's voice wave which lips pattern and microphone 2 of the 
utterance person whom the camera 1 photoed collected is beforehand registered into the data 
registration section 5. The voice wave which carries out speech recognition in the lips pattern 
and the speech recognition section 4 which carry out pattern recognition to the contents 
registered into the data registration section 5 at the time of sending out of a news program in 
the pattern recognition section 3 is collated in the data collating section 7. By changing one side 
into the signal for broadcast with a signal converter 10 among the clause information saved in 
the clause preservation section 6 according to the collating result, and the clause information 
changed by the clause transducer 9, and superimposing on a news sending-out signal with the 
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signal composition vessel 1 1 The detailed contents of news can be transmitted to real time to a 
hearing-impaired person, without needing the expert of sign language. 

[0065] Moreover, the recognition result of the pattern recognition section 3 to an utterance 
person's lips pattern which the camera 1 photoed, and the recognition result of the speech 
recognition section 4 to the voice of the utterance person whom the microphone 2 received are 
beforehand registered into the data registration section 12. The recognition result of the 
contents registered into the data registration section 1 2 at the time of sending out of a news 
program and the pattern recognition section 3 and the recognition result of the speech 
recognition section 4 are collated in the data collating section 14. Also when one side is changed 
into the signal for broadcast with a signal converter 10 among the clause information saved in 
the clause preservation section 13 according to the collating result, and the clause information 
changed by the clause transducer 9 and it superimposes on a news sending-out signal with the 
signal composition vessel 1 1 The detailed contents of news can be transmitted to real time to a 
hearing-impaired person like the above, without needing the expert of sign language. 
[0066] Furthermore, the clause information which changed the recognition result of the pattern 
recognition section 3 to an utterance person's lips pattern and the recognition result of the 
speech recognition section 4 to an utterance person's voice by the clause transducer 15 is 
beforehand registered into the clause registration section 1 6. The contents registered into the 
clause registration section 16 at the time of sending out of a news program and the head part of 
the clause information changed by the clause transducer 15 are collated in the clause collating 
section 17. Also when one side is changed into the signal for broadcast with a signal converter 
10 among the clause information registered into the clause registration section 16 according to 
the collating result, and the clause information changed by the clause transducer 15 and it 
superimposes on a news sending-out signal with the signal composition vessel 1 1 The detailed 
contents of news can be transmitted to real time to a hearing-impaired person like the above, 
without needing the expert of sign language. 
[0067] 

[Effect of the Invention] As explained above, according to the news sending-out equipment for 
hearing-impaired persons of this invention The lips pattern and voice which were beforehand 
registered with an utterance person's lips pattern and voice at the time of news sending out are 
collated. By changing one side into the signal for broadcast among the text which recognizes and 
obtained the text which recognizes and obtained an utterance person's lips pattern and voice at 
the time of news sending out according to the collating result, the lips pattern registered 
beforehand, and voice, and superimposing on a news sending-out signal It is effective in the 
ability to transmit the detailed contents of news to real time to a hearing-impaired person, 
without needing the expert of sign language. 

[0068] Moreover, while registering an utterance person's lips pattern, and a voice wave, 
respectively according to other news sending-out equipments for hearing-impaired persons of 
this invention The text which recognizes and acquired the lips pattern and a voice wave is saved. 
The lips pattern of the utterance person who reads out the contents of news at the time of 
news sending out and a voice wave, the registered lips pattern, and a voice wave are collated. By 
changing one side into the signal for broadcast among a lips pattern and an audio recognition 
result, and the saved text according to the collating result, and superimposing on a news 
sending-out signal It is effective in the ability to transmit the detailed contents of news to real 
time to a hearing-impaired person, without needing the expert of sign language. 
[0069] Furthermore, according to another news sending-out equipment for hearing-impaired 
persons of this invention, recognize an utterance person's lips pattern, and an utterance person's 
voice, respectively, change into text, and it registers, respectively. The lips pattern of the 
utterance person who reads out the contents of news at the time of news sending out and an 
audio recognition result, and the registered text are collated. By changing one side into the signal 
for broadcast among a lips pattern and an audio recognition result, and the registered text 
according to the collating result, and superimposing on a news sending-out signal It is effective 
in the ability to transmit the detailed contents of news to real time to a hearing-impaired person, 
without needing the expert of sign language. 
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[0070] According to still more nearly another news sending-out equipment for hearing-impaired 
persons of this invention, further again Change into clause information the text which has 
recognized an utterance person's lips pattern, and an utterance person's voice, respectively, and 
obtained them, and it is registered, respectively. The clause information which changed and 
obtained the lips pattern of the utterance person who reads out the contents of news, and the 
audio recognition result at the time of news sending out, and the registered clause information 
are collated. By changing one side into the signal for broadcast among the clause information 
which changed and obtained the lips pattern and the audio recognition result according to the 
collating result, and the registered clause information, and superimposing on a news sending-out 
signal It is effective in the ability to transmit the detailed contents of news to real time to a 
hearing-impaired person, without needing the expert of sign language. 
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* NOTICES * 

JPO and NCIPI are not responsible for any 
damages caused by the use of this translation. 

1 This document has been translated by computer. So the translation may not reflect the original 
precisely. 

2.**** shows the word which can not be translated. 
3.1n the drawings, any words are not translated. 
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[Drawing 2] 
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